Papers with fine-tuning settings
Practical Transformer-based Multilingual Text Classification (2021.naacl-industry)
Copied to clipboard
| Challenge: | XNLI does not reflect the data availability and task variety of industry applications. |
| Approach: | They compare transformer-based text classification methods to multilingual models in five different languages . they use a task- and domain-adaptive pretraining and data augmentation technique . |
| Outcome: | The proposed methods outperform monolingual models on two tasks in five languages . the results show that practical modifications can improve model performance without labeling . |
Contextualizing Language Models for Norms Diverging from Social Majority (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies on transformer-based language models have shown that there seems to be a 'moral dimension' to LMs, as they show high accuracy in related downstream tasks such as moral reasoning and action classification. |
| Approach: | They propose a mechanism based on deontic logic to allow for a flexible adaptation of individual norms by de-biasing training data sets and a task-reduction to textual entailment. |
| Outcome: | The proposed mechanism de-biases training data sets and reduces tasks to textual entailment. |
LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error (2024.acl-long)
Copied to clipboard
| Challenge: | Existing work on tool-augmented LLMs focuses on the broad coverage of tools and the flexibility of adding new tools. |
| Approach: | They propose a biologically inspired method for tool-augmented LLMs that orchestrates three key mechanisms for successful tool use behaviors in the biological system: trial and error, imagination, and memory. |
| Outcome: | The proposed method improves tool learning for LLMs under both in-context learning and fine-tuning settings, bringing a boost of 46.7% to Mistral-Instruct-7B and outperforms GPT-4. |
LexCLiPR: Cross-Lingual Paragraph Retrieval from Legal Judgments (2025.acl-long)
Copied to clipboard
| Challenge: | Existing work on IR focus on retrieving entire cases rather than precise, paragraph-level information. |
| Approach: | They propose a cross-lingual dataset for paragraph-level retrieval from ECtHR judgments . they evaluate retrieval models in a zero-shot setting and use multilingual case law guides . |
| Outcome: | The proposed model excels in cross-lingual retrieval, while siamese architectures are better suited for monolingual tasks. |
SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval (2024.findings-acl)
Copied to clipboard
Siwei Wu, Yizhi Li, Kang Zhu, Ge Zhang, Yiming Liang, Kaijing Ma, Chenghao Xiao, Haoran Zhang, Bohao Yang, Wenhu Chen, Wenhao Huang, Noura Al Moubayed, Jie Fu, Chenghua Lin
| Challenge: | Multi-modal information retrieval (MMIR) is a rapidly evolving field . current benchmarks for image-text pairings overlook the scientific domain . |
| Approach: | They develop a scientific domain-specific MMIR benchmark to evaluate image-text pairings using open-access research paper corpora. |
| Outcome: | The proposed benchmarks are based on 530K image-text pairs extracted from scientific documents with detailed captions. |